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(54) Method and system for dispatching client sessions within a cluster of servers connected to 
the World Wide Web 
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(57) A method and system for preserving load bal- 
ancing of the client transactions, for the whole duration 
of the client sessions, in a Web site implemented in the 
form of a cluster of servers is described. The invention 
manages to send only the initial request of each client 
session to the site load balancer thus, greatly enhancing 
the capability of the site to accept new session requests. 
All subsequent requests from a client are forwarded di- 
rectly to the server first selected so that the sessions 
cannot be later broken by the load balancer. The 
scheme works regardless of the fact that the client is 
beyond a proxy or a firewall and greatly contributes to 
the performance of the Web site. 
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Description 

Field of the Invention 



[0001] The present invention deals with the Global In 
erne, network and more particularly , 0 those o the In" 
ternet Servers of World Wide Web (UvK£~ 



7c2TZ ^ at,0n,r0mtheco -Panysexis,in 9 ap- 
P cations. Th.s could include checking the availa 
b.l..y of products, querying bank account bafanc es 
or searching problem databases. These appTca 
ons req ul re actual processing on the server 
tem to dynamically generate the Web page This 

dramfcauy the processing "power S 

quired in the server. 



Background of the Invention 

[0002] The Internet isthe world's largest network and 

ZlT™, eSSen,ia ' h 0 ^atiL such as gov 
emment, academia and commercial enterprises Trans 

*2 ,n,e ™ - more com™ 

formation or service available for their customers is a 
reasH ** ^nization. Howevef 

inn ° Va,i0n P ° ten,ial benefits pro 
v-ded by a company's Internet solution, its value is oreat 
<y reduced if the information cannot be acces edTn a 
reasonable response time accessed »n a 

[0003] The load on an Interne, site is unlikely to re 
main constant. The number o, accesses on a Web sen, 
er can increase for several reasons. 

1. Most companies add their Web site's address to 
televison, «aToandp^ao>^^ ana - < S^ i ; 

or TZu? ^ Slte iS Pr ° Vidin9 Use ' u ' Nation 

4. Most Web sites begin simply, with fairly modest 
content, mostly text, with some images. As 
doners grow in confidence, more resources 

the,r modem speeds, most sites move towards rich 
er content. Thus, not only do hi, rates increase Z 
the average data transfer per hit also rises 
5. Most sites begin as presence sites providing cor- 

KSTJ? on ,he ,n,eme ' and ma ^ ^ 

ion about the company available to potential cus 
tomers^os, present si.es use predominant a{ c 

25c'™ Markedup Langua9e or H ™ L W 

o^tsk Th6 S r e 9enera,ed i0 adVanC9 3nd Sto ^ 
on d,sk. The server s.mpV reads the page from the 

d.sk and sends it to the browser. However manv 

companies are now moving towards integral ioTap 

Pl-cations that a.bw users of .he Web site ,o ££, 



o, a^ n,em e f !f Vera ' ™" ,0deal *. growth 
Jh , ? S ' ,e " ke PUrChasin 9 an system that 
b much too large. This is one way to deal with Web site 
growth; however, mos, companies are no, w ,1 tot 

than they requ ire particularly since the benefits maUhev 
w.l, gam from the site have yet to be provTlf^Z 
o purchase a minimal initial system and to upg adl * 
1 InTh" ,h ? fte dem0nst — * -orth tot e CO m 
» mull 1 m ° f S °' Uti0nS ,0ad ba,an ™9 between 
m^pleserversisveryoftenused .In, hiscase theload 

This aii >Vera Site ' S Da,anced between multiple servers^ 

available from a s.ngle system and allows for easv ud 
grading by simply installing addjtjona| J™*" Z 
reconfiguring the cluster to use the additionaUe^T 
This solution can also provide the added benefit of hioh 
e server availability. The load-balancing softly 

so 2 \ T ,hel0adbe,wee "«heremaining S i, es B^causf 

n a S1 ng le entlty , jmp(jes ^ ^ ^ 
formed between these servers so as to actually ac ieve 
optimum performance. A discussion on this and ZZ 

Served Diai t T ^ Highly Avai,able w * 

?omn? c rma, ' 0n Su P erhi 9hway, Forty-first IEEE 

SS e 9 r 11 ^ ,e, ^ nterna,i0nal Co ^rence (Cat. No 
96CB35911), pp. 85-92, Feb. 1996 

0005] Load-balancing products have made their wav 
to he ma rk t lBM>s me way 

^^^^^«^«^ 

' , S,0n 01 hav,n 9 J"st one server by grouping svs 

- SL^rr a c,uster ,hat ^ - ^ 

virtual server. The service provided is no longer tied to 
a specific server system; so it is possible to addofre 

fTm e a fn y ,f ^ fr ° m C ' US,er ' °' Shutdo - « 
or maintenance whi,e mainlining continuous servS 

dress of the a Network Dispatcher machine, which de- 
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cides with each client request which server is the best 
one to accept requests, according to certain dynamically 
set weights. Network Dispatcher routes the clients' re- 
quest to the selected server, and then the server re- 
sponds directly to the client without any further involve- 
ment of eND. This makes it possible to have a small 
bandwidth network for incoming traffic (like Ethernet or 
token ring) and a large bandwidth network for outgoing 
traffic (like ATM - Asynchronous Transfer Mode or FDDI 
- Fiber Distributed Data Interface or Fast Ethernet). It 
can also detect a failed server and route traffic around 
it. Genera! information on the way of performing load 
balancing between multiple servers and on eND product 
can be found in a 'Redboook* by IBM published by the 
Austin, Texas center of the International Technical Sup- 
port Organization (ITSO) and untitled "Load-Balancing 
Internet Servers" under the reference SG24-4993 on 
December 1997. 

[0006] Those products are great to achieve what they 
have been designed for, i.e., load-balancing and indeed 
allow to build scalable Web site capable of coping with 
a rapidly growing demand for higher traffic. However, 
they have created their own difficulties too. Because 
there are now numerous sophisticated Web servers that 
allow to handle dynamic Web pages they need to be 
session-aware for every user accessing their service. 
Several techniques indeed exist to keep track of the con- 
text in which a particular user is accessing a Web server. 
They are of two kinds: 

• the contextual data is circulating, back and forth, in 
the IP packets exchanged between the client and 
the servers. For example, it can be part of the Web 
pages themselves. 

• or the contextual data is kept in the Web server ac- 
tive memory or on disk. This second solution is nec- 
essary whenever the amount of data needed to de- 
fine each session context is too large to be practi- 
cally transported over the network with each trans- 
action between the client and the servers. 

[0007] Then, load-balancing products such as eND 
manage not to dispatch randomly the traffic to the serv- 
ers of their cluster. They keep track of the user requests 
which must end up into the same server while a session 
is active. To achieve this, the usual technique, well 
known from the art, consists in utilizing the IP address 
of the client. Then, each transaction coming from the 
same IP address is dispatched to the same server. 
[0008] However, this does not fit in the now frequent 
situations where the end user and the server are on ei- 
ther side of a proxy, socks or fire-wall. All those devices, 
part of the Internet, are intended to deal with specific 
problems like, for example, the isolation of an intranet 
that must not be freely accessible by outsiders without 
any control thus, leading to place a fire-wall at the in- 
tranet gateway. Or a proxy, so as the users within an 
intranet are seeing the whole Internet through a com- 



mon gateway device, somehow caching it, in an attempt 
to achieve overall better performance. In these situa- 
tions, the client IP address is not actually known by the 
network dispatcher which establishes in fact a TCP con- 
s nection (the Transport Control Protocol of the Internet 
protocol suite) with the proxy, the socks or the fire-wall 
rather than directly with the end-user. Therefore, the net- 
work dispatcher is no longer session aware that is, has 
no information that would allow it to decide that a par- 
10 ticular end-user, for example located beyond a proxy, 
that was engaged into a transaction such as buying a 
product from a virtual shop, an application that was first 
selected by the dispatcher on a particular server in the 
cluster of servers, has not yet completed. Then, further 
15 requests from the end-user, sometimes occuring after a 
long pause, could be dispatched differently by the net- 
work dispatcher just because it does its job of balancing 
the traffic towards a less busy server within the cluster 
with the obvious consequence that the new server is not 
aware of the transaction in progress. 
[0009] And there is another undesirable effect of hav- 
ing the end-users beyond a proxy for a load balancer. 
All the individual users within a group, for example an 
intranet then, appear to the load balancer as a single 
user because their IP address is the same since it is the 
one of the proxy or fire-wall. Therefore, the load balanc- 
er which tends to maintain the dispatching of a given 
user towards the same server, in an attempt not to break 
sessions, at least while an inactivity timer has not 
elapsed, keep sending the traffic of the whole intranet 
to the same server. This seriously goes against what this 
kind of product is trying to achieve, i.e., load balancing. 
Although the individual users within a group would cer- 
tainely enjoy not being served by the sometimes same 
busy server, because they are seen as being a single 
client by the load balancer, it is no longer possible to 
discriminate the individual users. 

Object of the Invention 

[0010] Thus, it is a broad object of the invention to 
overcome the shortcomings, as noted above, of the prior 
art and therefore enabling a particular server, within a 
cluster of servers, to continue serving a given end-user 
while the current session is active and being able to dis- 
criminate the individual users within a group (intranet) 
so as to maintain a good load balancing over the cluster 
of servers. 

[0011] It is a further object of the invention to improve 
the efficiency of the load balancer by requiring oniy one 
interrogation per session thus, freeing it to dispatch 
even more transactions over the cluster of servers. 
[0012] Further advantages of the present invention 
will become apparent to the ones skilled in the art upon 
examination of the drawings and detailed description. It 
is intended that any additional advantages be incorpo- 
rated herein. 
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of «h.Lj^csrssi? ,he who,e dura,i ° n 

ity of servers and tol^Z?^**"* 
Tom a p,urali,y of c^XZc^T™ a ° C888ad 

request is forwarded to the select « . J 3 ' 

'-..ha. ml itr m : works regard,ess « - 

Brief Description of the Drawings 
[0017] 

H9ure 1 Describesthepriorartwherealoadbatanc- 
er « d,spatchin9 end-user requests ovTa 
cluster of servers however, being pZtnt 
ed -of always fully taking advantage oUhe 
computng resources of the servers when 

Fiaure 2 n ^ ,0 ° ma " y rec > ues,s * Progress 
F.9ure 2 Dep.cts the common case when" Proxy or 

users and the cluster of servers thus ore- 

Patch.ng o, the incoming requests. 2 

the breakmg of end-user sessions in case 
of mactrvity is described 

9Ure 3 ,t SCribes the 9eneral solution brought by 
the .nventron to the shortcomings of the 
previous art 

^ure 4 describes a p a r, icu , ar implementatjon 

7 ° f ,he work| oad over all the servers in a 
'W 5 dsscnbes an „ emale , mplemm , ata „, 



In. invention rtlh , he same advances. 

as a angle, virtual server so that thJ 11 ^ Ves 
address ,„ ,™ WSm) ™™ and ,p 

a source and a rw, . hese Packets have 

dress Te'p SET? J*"* ^ deStina,ion ad ' 
* « the CusTe i , OS 2 ''T P A " " v « 
•P address ^^SS'SlSr T* T 
dispatcher system checks whth SS t0 °' The 

routes the packeHn .hit ^ ' S less bus V and 

=r ^:sre== 

'eceived „c™ on. S '„ ^ SS 

even though the reques i s »^ ? f ° rexam P ,e 1240], 

Processedbytheclusw. f ame type of service 
oy the cluster of servers [230] there is a good 
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chance that a continuous flow of requests arrives to the 
load balancer. Then, load balancer is bound to deliver 
the requests to the initial selected server even if other 
servers of the cluster, not as busy, could deliver the 
same service. The above case is not unlikely to happen $ 
just because the intranet is shared, e.g., by the person- 
nel of a specific company say, a financial institution. All 
members may have some interest to consult the same 
type of information during the day say, stock exchange 
rates. Therefore, this kind of situation may tend to pre- 10 
vent load balancer to spread an equal share of the work- 
load over all servers of a cluster of servers. 
[0020] A second type of problems is encountered if, 
on contrary of the here above just described situation, 
no request is arriving for some time to the load balancer 15 
so that a significant period of inactivity let think to the 
load balancer that the user session has ended. Then, it 
may decide to reassess load balancing with the arrival 
of another request even though the session is still in 
progress at the viewpoint of the end-user. This, may 20 
happen when a particular end-user is pausing for a long 
time while the other users on the intranet are not ac- 
cessing the cluster of server [230]. Therefore, a further 
request from the end-user [260] may end up in a differ- 
ent server of the cluster, i.e., not in [240]. The new se- 2s 
lected server is not aware of the transaction in progress 
and the context is lost. Hence, a transaction that in 
progress, e.g., the payment of an item bought from a 
virtual shop is aborted. 

[0021] Figure 3 depicts the general solution to the 30 
problems induced by the use of a load balancer dis- 
patching incoming Web browser requests over a cluster 
of servers as discussed in the two previous figures. 
Whenever a new request [300] from a Web browser 
[305] is issued it is forwarded to the load balancer [31 0]. 35 
This, because it is load balancer DNS name and corre- 
sponding IP address (referred to as DNS0 and IPO [31 2] 
in the particular example of figure 3) which is made pub- 
lic for the service or the set of services advertised for 
the Web site implemented in the form of a cluster of serv- 40 
ers [320]. On contrary of the previous art it makes no 
difference for the invention of receiving the request ei- 
ther directly [326] or through a proxy [325]. Whenever 
the initial request reaches the load balancer [311] it is 
dispatched to one of the servers of the cluster of servers 45 
[320]. The decision of routing towards one particular 
server, like [313] in this example, is the prime job of the 
load balancer. The metric used to decide which server 
is to be selected at a given instant depends on the de- 
sign of the load balancer which is assumed to collect s° 
from all the servers, at regular intervals, performance 
information regarding their level of activity. In broad gen- 
eral terms it can be said that the less busy of the servers 
is selected in an attempt to indeed reach the goal of bal- 
ancing the workload equally over all the servers. The 55 
invention does not, per se, interferes with this process 
which is under the sole responsibility of the load balanc- 
er. However, it contributes dramatically to improve the 



307 A1 




job of the load balancer by forwarding to it only the initial 
requests, like [300], issued by the Web browser [305] 
when initializing a session as this will become apparent 
in the following. Thus, the request is forwarded, in the 
example of figure 3, to server [321 ] that met the criterion 
for being elected to process initial request [311] at time 
it reached the load balancer [310] on the basis of the 
performance data that were collected by the load bal- 
ancer from the servers. At this point the rest of the ses- 
sion is going to be handled solely by the particular serv- 
er, e.g., [321] without any further implication of the load 
balancer which is then completely free to accept all new 
requests arriving to the site and generally referred to as 
-hits" in the literature that deals with Web site perform- 
ances, at least until not all the resources of the cluster 
of servers are completely exhausted. This is a complete 
departure from the prior art where the processing of the 
new hits interfered with the processing of all the ses- 
sions already in progress as illustrated in previous fig- 
ures thus, leading to postpone the processing of a new 
request by the load balancer when it is too busy itself 
dispatching the numerous requests of the sessions al- 
ready in progress, even though there may have still 
plenty of computing resources available hence, wasted 
in the cluster of servers. The above is made possible 
because each of the servers within the cluster of servers 
has its own unique DNS name et corresponding IP ad- 
dress. For example DNS1 and IP1 for the server [321]. 
Therefore, the server that has been elected by the load 
balancer, upon receiving the initial request [313] will re- 
ply directly [330] to the Web browser of the end-user. 
This reply mentbns the Uniform Resource Locator or 
URL to be used for the further requests of that session. 
This URL contains the DNS name or the IP address of 
the elected server i.e.: DNS1 orlP1. All further requests 
[332] are forwarded directly to the selected server thus, 
freeing the load balancer of dispatching the subsequent 
session requests as mentioned above and insuring that 
all are going to reach the same server for the whole du- 
ration of the session. Again, the Web browser and the 
cluster of servers may be on either side of a proxy like 
[325] without impacting, at all, the above scheme on 
contrary of the previous art. 

[0022] Figure 4 illustrates one particular implementa- 
tion of the general solution depicted in figure 3. It takes 
advantage of the here above mentioned option that a 
specific response could be issued by the selected server 
for informing the end-user browser ol the actual DNS 
name of the server elected to process its session. The 
protocol, part of the TCP/IP suite of protocols, used by 
the Web server to transfer hypermedia documents 
across the Internet, known under the acronym of HTTP 
for Hyper Text Transport Protocol, specifically foresee 
the possibility of redirecting a request that was issued 
for a specific DNS name to another DNS name for the 
duration of the session. Hence, whenever the selected 
server [400] is receiving a new request [410] it has just 
to respond with a HTTP redirect command [420], either 
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http://dnsl/links.htm|\ 

^ ' 

<html> / 

<title>F(K>BAR LINKS</title> 
<body^ 

<h1>FOOBAR LINKS</h1> 

<a ttref=" http:/A/vww.ietforg/ ">IETF</a> 

<a/nref=" http://www.w3.org/">W3C</a> 

J^y^ /http://dns1/foobar.html \_ 

7html> 




Server 1 
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<html> 

<title>FOOBAR</title> 
<body>^^^^ 

<h1 >FOOpAfrP7«3E^/h1 > 
<a href="/4inks.htmr'>fiobar links</a> 

</body> 
</html> 



/http://dnsO/index.html \_ 



fhttp://dns2/(inks.html)^ 



<html> / 
<title>FOOBAR LINKS</titIe> 
<body> / 

<h1>FOOBAR LINKS</h1> 
<a href=/ http://www.ietf.org/ ">IETF</a> 
<a hrett' http://www.w3.orgr>W3C</a> 

<fbo&j> / htt P ://dns2/foobarhtml V 

L </html> 



<html> 

<title>WELCOME</title> 
<body> 

<h1>WELCOME PAGE</h1> 
<a href="http://drjs1 /fbobar.html">foobar</a> 

. ../... 
</body> 
</html> 
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<hlm> 

<title>FOOBAR</title> 
^6ody>* 

<h1 >F006ftR-P3%G^</h1> 
<a href= w (iinks.htmP^oobar links</a> 

yhttp://dnsO/index.html \ 

</html> 
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<html> 

<title>WELCOME</trtle> 
<body> 

<h1>WELCOME PAGE</h1> 
<a href="http://dns2/foobar.htmr , >foobar</a> 
„./... 
</body> 
</html> 
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